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ABSTRACT 


This  paper  describes  a numerical  evaluation  technique  for 
weighting  multiple'  options.  It  is  a ver*r  simple,  but  useful, 
technique  for  sys temati call y applying  relative  weights  to  an" 
number  of  options.  To  make  the  nrocess  clear  and  understand- 
able to  a vn.de  variety  of  readers,  the  technique  is  applied 
to  the  purchase  of  a new  automobile.  A numerical  evaluation 
of  the  optional  equipment  on  an  automobile  is  made. 

The  numerical  evaluation  technique  map  be  applied  to  a 
wide  variety  of  multiple-option  problems.  The  technique  can 
be  applied  to  weight  multiple  objectives,  requirements,  pri- 
orities, methods,  or  practically  ^.n-  set  of  attributes.  Sub- 
jectivity is  not  eliminated  b"  the  nrocess,  but  it  is  syste- 
matised to  allow  a uniform  and  unbiased  approach.  It  has 
important  application  in  the  s "stems  analysis  field,  parti- 
cularly in  weapons  s vs terns  evaluations,  requirements  analysis 
and  lo^i s ti crl  analvnir . 


I.  IKTRODUCTIOK 


Systems  ar.  .l*vis  largely  involves  the  task  of  assisting 
decisionmakers  in  choosing  preferred  future  courses  of  action 
from  several  alternatives.  Maximum  use  is  made  of  Quantita- 
tive techniques . Often,  however,  quantitative  analysis  is 
not  feasible . Man  * intangible  alternatives  must  be  anal-ved 
in  a subjective  manner.  Subjective  analysis  involves  value 
judgements  by  the  decisionmaker  and  thus  is  specific  to  the 
individual  performing  the  annl*vir . This  does  not  mean  that 
the  subjective  evaluation  must  be  capricious,  although  it 
often  is.  A subjective  anal* vis,  although  variable  among 
ini divi duals  performing  the  anal-vis,  can  still  be  self consi- 
stent a nd  s-s t emati c . 

Whether  v*e  realise  it  or  not,  vre  all  perform  systems 
anal-sis  in  our  everyday  lives.  For  most  people,  the  techni- 
cue  is  one  of  subjective  analysis  vdth  limited  quantitative 
inruts  (usuall-  cost)  also  considered . V/e  are  often  con- 
fronted vdth  the  necessity  of  choosing  among  several  alter- 
natives and.  man-  combinations  of  alternatives.  Tving  to 
anal— e too  man:*  variables  at  once  becomes  a bewildering 
task.  The  frustration  of  grappling  vdth  comrlex  choices 
invites  one  to  "give  up"  and  make  the  decision  in  a h-'-rhasard 
manner.  Such  a decision  usuall-  giver:  1 a sr- than- optimum 
results  and.  often  vinos-  up  costing  the  decisionmaker  mono  - 
and  inconverd once  •.'hick  he  can  ill  afford.  Such  problems  can 
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'be  minimi vcdby  usinr  a ryotenntic  approach  to  a subjective 
evaluation.  of  multiple  options.  It  is  th"  "0-1  o:n  this 
paper  to  describe  one  ouch  approach. 

II.  TH3  PROBLEM 

A common  decision  involving  several  options  and  combina- 
tions of  options  is  the  determination  of  the  optional  equip- 
ment to  include  in  the  purchase  of  a new  automobile.  Cl earl" 
the  decision  must  be  subjective..  Options  that  are  important 
to  one  burner  may  be.  completely  unimportant  to  another.  This 
is  the  vor”  reason  that  the  equipment  is.  optional  - - - to 
allot7  the  buyer  to  choose  the  combination  which  subjectively 
suits,  his  needs. 

Suppose  the  buyer,  Mr.  Jones,  has  decided  that  he  wants 
to  bus-  an  intermediate  sise  sedan.  The  basic  car  comes 
equipped  with  a.  six  cylinder  enyine,  standard  manual  trans- 
mission and  precious  little  else.  The  manufacturer  lists 
about  twenty  options  which  can  be  added,  at  additional  expense, 
to  the  basic  car . Sunnose  Mr.  Jones  is  onl"  interested  in  It 


of  those  fC  options.  Some 
much.  Some  he  wants  ver~  : 
about.  Mr.  Jones  is  no  ma 
all  14  costs  and  their  rel 
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options  cost  little  and  some 
much,  and  others  he’s  not  so 
thorns ti cal  wirwrd.  and  cannot 
stive  irroortance  in  his  mind 
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options  is.  given  a key  letter.  These  are  just,  assigned 
secuentiall”,  and  the  order  maker  no  difference.  Next,  the 
systems  analyst  makes  up  an  evaluation  matrix  using  the  ke;-” 
letters.  The  matrix  is  made  uo  so  that  one  scuarc  exists 


for  every  possible  pairing  of  key  letters, 
the  basic  matrix -prepared  Cor  Mr. Jones.  The 
Figure  1 contains  the  ootions  and  key  letter 
half  the  her  letter  matrix.  Ignore  the  ’k;ei 
for  the  oresent. 
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III.  THE  NUMERICAL  EVALUATION 
With  Figure  1,  the  analyst  is  nor  reads  to  have  Mr.  Jones 
make  the  evaluation.  Starting  vtith  ke**  letter  "A",  Mr. 

Jones  is  asked  to  compare  that  option  (S  evlinder  engine) 
to  each  of  the  other  options.  He  is  asked  to  indicate, 


first,  his  preference  between 
secondl",  the  extent  of  his  ore 
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3 CYLINDER  ENGINE 
AU  TOMATI C TRAN SMI  S SION 

traction  ' type  diftbhsntial 
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give  it  c'ome  thought,  hut  cleahL"  prefers-  one  alternative,, 
then  he  "ou.l^  '-s,~i~h  r n2".  I"  he  has  ° ver  * -''•"iev.lt 

time  deciding,  then  a "1"  would  be  assigned.  The  ore f erred 
option  is  named  b’?  its  ke-  letter  followed  bv  the. extent 
of  preference.  For  example  a "3-1"  in  the  A-B  intersection 
block  of  Figure  1 means  that  in  comparing  options  A and  B, 
Mr.  Jones  preferred  options  B but  was  not  very  sure  about 
his  decision.  Repeating  this  process  for  the  entire  matrix 
recuired  Mr.  Jones,  to  compare  each  option  .against  every 
other  option.  None  could  be  skipped  and  during  the. process 
it  would  be  very  difficult  for 'Mr.  Jones  to  anticipate  the. 
final  outcome.  The  completed  matrix  is  shown  in  Figure  ?. 
The  "weight"  column  in  Figure  2 is  the  desired  output  of 
the  evaluation.  The  weights  are  simply  the  summation  of 
the  preference  ratings  for  each  key  letter  obtained  by- 
adding  all  the  numbers  associated  with  each  respective  key 
letter.  For  example,  in  Figure  ? there  was  a total  of  ?9 


"preference  points"  associated  vrith  kc"r  letter  "A". 

The  weights  obtained  from  the  matrix  evaluation  process 
represent  the  relative  preference  that  Mr.  Jones  had  for 
the  fourteen  options.  Examination  of  the  weights  shows 
that  Mr.  Jones  has  the  strongest  preference  for  an  auto- 
matic transmission,  follovred  b * an  eight-c”linder  engine, 
and  then  b_r  air  conditioning.  The  aero  ratine-  on  the 
power  antenna  shov-s  that  he  preferred  ever"  option  to  it. 
With  the  weights  in  Figure  ?,  Mr.  Jones  now  has  a credible 
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NUMERICAL  EVALUATION  TECHNIQUE 


Name  Mr.  Jones 


Date  1/2/7 5 Project  New  Auto 


Evaluation  Summary 


Kcy 

t-ITTCR 


Options 

0 CYLINDER  ENGINE 
AUTOMATIC  TRANSMISSION 
TRACTION  TYPE  DIFFERENTIAL 
HEAVY  DUTY  SUSPENSION 

DISC  BRAKES 

FOYER  STEERING 

FOYER  WIN 5Q17 5 

FOYER  DOOR  LOCKS 

PCY5R  ANTENNA 

AIR  CONDITIONER 

AM  RADIO 

FM  RADIO 

AUTOMATIC  SPEED  CONTROL 
SUN  ROOF 


Weight 


BCD  c F G HI  J KL  MN 

I I I | | T 1 1 I I I 1 

>B-1  A- 3 A- 2 A- 2 A-I  A- 3 A-3  A- 3 A-3|  A- 2 

B 

B-3  B-2  B-2) B-2  3-3  B-3  B-3  B-l  B-3  B-3  B-3  B-3 

C 

D-l  E-2  F-3  C-3  C-3  C-3  J-3  C-l  C-l  C-2  C-3 

0 E-2  F-2  D-3  D-3  D-3  J-2  D-l  L-l  D-2  D-3 

E F-l  E-3  E-3  3-3  J-l  E-2  E-l  E-3  E-3 


NUMERICAL  EVALUATE 


F-3  F-3  F-3  F-l  F-2  F-2  F-3  F- 


Notc:  — Evaluation  V/cigmy  Factors 

1 m Minor  oiFrcuENCs  in  importance 
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3 n Major  oirrtKEMce  in  importance 


G— 2 G-?  J-3  K-2  L— 3|  M-2  K-l 


H-2  J-3  IK-3!  L— 31  M-; 


J-3  K-3  L-3  M-3  N-3 


Figure  2.  Completed  Evaluation  Matrix 
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priority  lift  of  options  and  using  just  thir  list  alone, 
if  in  •"  much  hotter  position  to  m-ke  rational  choicer.  If 
he  had  a.  limit  on  his  funds,  he  could  select  those  options 
;vdth  the  highest  weight  first,  repressing  domn  the  priority 
list  until  his  funds  v/ere  expended.  Thus,  Mr.  Jones  is  able 
to  make  rational  and  relatively  painless  decisions  on  the 

i f 

options  without  his  having  the  napping  feeling  that  he  had 

i 

overlooked  something. 

The  example  could  be  pushed  a step  further  to  more  clearly 
consider  the  costs  o”  the  options . Some  options  are  ver-* 
expensive  v/hile  others  are  relatively  inexpensive.  With 
limited  funds,  can  Mr.  Jones  pet  the  most  satisfaction 
from  one  expensive  option  or  from  several  inexpensive  options? 
If  v;e  consider  the  weights  as  indicators  of  Mr.  Jones's  re- 
lative utility  (or  satisfaction) , then  e/e  can  see  hot/  many 
"utility  points  per  buck"  he  is  getting  for  each  option. 

Table  1 shov/s  these  values  calculated  for  the  14  options 
in  the  example . The  ratios  are  multiplied  bv  100  for  con- 
venience because  it  is  easier  to  deal  with  v/hole  numbers 
than  with  fractions. 

Examination  of  Table  1 shows  a suite  differenct  picture 
than  just  considering  the  weights  b_r  themselves.  It  can 
be  seen  that  the  heav-  duts*  suspension  provides  the  most 
utilit-  per  buck,  folio- md  b"  disc  brakes  and  the  trf ction- 
tvoe  different!'--! . What  does  this  moan?  Mr.  Jones  still 
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Table  1.  Utilitv  Rates  for  Each  Option 


OPTION 

WEIGHT. 

■m 

UTILITY 
PER  DOLLAR* 

8 Cylinder  Engine 

09 

1?8 

03 

Automatic  Transmission 

3? 

027 

14 

Tracti on-Tvpe  Di f f erenti al 

Id 

50 

as 

Heavy  Duty  Suspension 

Id 

31 

Disc  Brakes 

03 

51 

45 

Power  Steering  . 

06 

118 

00 

Power  W indovs 

/' 

10  A 

3 

Power  Door  Locks 

r\ 

S 

70 

. 3 

Power  Antenna 

0 

34 

0 

Air  Conditioner 

0(5 

;-.U 

414 

7 

AM  Radio 

11 

63 

18 

M Radio 

lc 

013 

9 

Automatic  Speed  Control 

10 

113 

9 

Sun  Roof 

6 

( 

461 

1 

¥<  Do  fined  an  100  :: 


prefers  an  automatic  transmission.  However,  if  hi?  fund  a 
are  vor*r  limited , he  would  ho  "citing  mors  utility  (or 
satisfaction)  tier  dollar  from  the  he?  • duty  suspension. 
Again,  this  gives  additional  data  to  Mr.  Jones  to  allot; 
him  to  make  a rational  decision  and  to  make  an  efficient 
allocation  of  his  resources.  The  evaluation  does  not  make 
the  'decision  for  Mr. Jonos,  but  it  does  gives  him  a system- 
atic and  rational  basis  for  making  the  decision. 


IV.  APPLICATIONS  TO  DEPTH S3 
The  new  car  example  shows  the  usefulness  of  the  numeri- 
cal evaluation  technique.  If  the  technique  was  onl~r  appli- 
cable to  bu'dng  a new  car  it  would  be  of  little  concern  to 
the  systems  analyst.  Besides,  one  fast-talking,  high- 
pressure  salesman  could  blow  the  whole  process.  It  does 
not  take  much  imagination  to  see  how  the  technique  can 
profitably  be  applied  to  defense  projects.  For  example, 
let  the  new  car  options  become  the  re  ui red  operational 
re  uirements  (ROC)  of  a combat  vehicle.  Let  Mr.  Jones 
become  the  ke”  managers  of  TACOM.  Then  the  process  would 


"iold  relative  we 
could  be  used  in 
c a ndi d at a vehi cl e 


rhts  of  the  various  ROC's.  The 
vsrietv  of  wa~rs.  For  example 
could  be  rated,  say  on  a scale 


100,  against  each  ROC.  Then  these 
tielied  b~~  the  weights  obtained  b‘* 


ratings  could 
the  numerical 
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technioue  and  summed  to  vield  a .weighted  ’'effectiveness 
r' cor°,f * Tl?  ^ p 0 c co^-r  c o p ^ d ^ ^ •f^]^ .o  iropo^'^-^p.c^  o ^ 

the  various  ROC ' as  well  as  the  ratings . The  scores  could 
then  be  used  in  a cost  effectiveness  study  of  the  candidate 
vehicles. 

Similarly,  the  technioue  could  be  applied  by  a contract 
proposal  review  committee  or  source  selection  board.  The 
options  could  be  various  attributes  of  the  contract  porposal. 
The  weighting  process  would  yield  the  relative  importance  of 
the  various  attributes.  Then  each  proposer  could  be  rated 
against  each  attribute  and  the  product  of  the  rating  'and 
the  weights  could  be  summed  for  a total  score. 

There  are  almost  a limitless  number  of  situations  in 
which  the  numerical  evaluation  technioue  could  be  applied. 

The  principal  strengths  of  the  technioue  are: 

1.  It  is  simple. 

2.  It  reouires  the  decisionmaker  to  consider  all  options. 

3.  It  is  systematic.  The  same  steps  are  always  involved 
and  proceed  in  a logical  and  unbiased  manner. 

1.  It  is  ver~r  difficult  to  manipulate  responses  to 
obtain  a preconceived  answer. 
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